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ABSTRACT 

The Student Response System (SSS) is discussed and 
evaluated* This is an electronic system whereby an instructor 
receives instant feedback from students by asking questions each with 
up to five multiple choice answers • The report covers a description 
Oi. the SRS, a review of the Skidmore College project on computer 
applications, and a review of the grant from the National Science 
Foundation, physical setup for the project, common uses of the SRS, 
uses of the SRS at Skidmore, combined uses of the SRS in class, uses 
of computer analysis, individualized uses of the response system, 
extent of usage, evaluation of the response system, a small 
statistical evaluation, and a summary of evaluation. (MJM) 
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STUDY AND EVALUATION OF THE STUDENT RESPONSE SYSTEM 
IN UNDERGRADUATE INSTRUCTION AT SKIDMORE COLLEGE 



Yu-kuang Cliu, PhD 



L DESCRIPTION OF THE STUDENT RESPONSE 
SYSTEM! 

A Student Response System, known as SRS 1000 C, 
has been developed in recent years by the Research 
and Development Center of the General Electric 
Company. It is essentially an electronic system 
whereby an instructor can get instant feedback from 
~ his students by asking questions each with up to five 
multiple -choice answers. There is an Input Station 
in front of each student, containing five white num- 
bered buttons corresponding to numbered multiple- 
choice answers. (See Fig. 1.) The student pushes 
one of the bu ttons to indicate his selected answer. 
If he wants to change his mind, there is a red but- 
ton to cancel his previous choice, which he can use 
before he puts in another answer. 

Student responses are scanned and reflected 
immediately in five percentage meters on the in- 
structor's panel mounted on "a lectern, showing him 
what percentage of the class has chosen each option 
under a given question. (See Fig. 2. ) There is a 
sixth meter to indicate the total percentage of the 
class that has answered that question. If there is a 
correct answer, its number may be shown in a "Dis- 
play Box" in fi^ont of the class. The five buttons can 
also be used as a five -step rating scale to express 
degrees of agreement, importance, interest, use- 
fulness, etc. I 

I 

It desired by the instructor, the Response System 
can be linked up with a teletypewriter, which will 
record and print out the student responses, identifi- 
able by seat numbers and question numbers, and at 
the same time will encode the data on a paper tape. 
The printout is immediately available after class for 
inspection by the instructor or students. (See Fig. 3.) 
The tape may be fed into the teletypewriter, when it 



is used as a terminal.on line with a computer. The 
computer will analyze the data according to a selected 
program and the results will be printed on the tele- 
typewriter. Such a printed analysis can be obtained 
in only a few minutes. 

II. SKIDMORE PROJECT ON COMPUTER 
APPLICATIONS AND GRANT FROM THE 
NATIONAL SCIENCE FOUNDATION 

The Student Response Sj'stem has been installed 
in recent years at several large universities, where 
large classes have found the equipment useful. But 
GE wanted' to explore the capability of this equipment 
in a small institution, and Skidmore College, which 
is a four-year liberal arts college located at Saratoga 
Springs, New York, with most classes enrolling 15 to 
25 students each, was interested in introducing com- 
puter applications into the teaching -learning process 
in undergraduate instruction. So the two organizations 
entered into a cooperative research project. The 
aim of the cooperative project is to try to use the 
SRS in as many different ways and academic disciplines 
as possible and to attempt an evaluation of its useful- 
ness in undergraduate instruction. 

Skidmore applied to the National Science Founda- 
tion for financial assistance (Proposal No. JO 00317) 
and received in October 1969 a grant of $70,000 
(Grant No. GJ-317) for the following budgeted ex- 
penses: 

Purchased Equipment $53,000 
Rental Equipment & Communications 15, 700 
Travel 300 
Expendable Supplies 1,000 
Total $70,000 
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Figure 1. Student Station 
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, Printout of Responses on a Teletypewriter, Format 
is the same for all kinds of questions. In this ex- 
aniplo» nine questions were used, and there were 16 

* students in the class. Responses are represented by 
numbers. The first column^ after the Line Numbers, 
consists of Instructor's Answers^ if questions have 
correct answers. Other wis e^ Instructor's response 
would be "0." Students' responses are identified by 
Scat Numbers. Some seats were not occupied. 
Student at Scat nfl failed to Answer the first three 
questions. Perhaps he was late to class. 

Figure 3. Raw Data Printout 

ThG anticipated contribution by Skid mo re to the 
support of the project, including Indirect Costs, 
was $78, 7G3, but this figure was actually exceeded 
by about $25,000 due to unexpected increase in con- 
struction costs. The purpose of the grant, as desig- 
nated by the NSF, was "Study and Evaluation of the 
Student Response System in Undergraduate Education." 
The period of the grant was originally for two years 
(October 1969 - October 1971), but due to the late 
start of the project caused by delay in construction, 
the termination date was subsequently extended by 
NSF, at Skidmore's request, to October 1972. How- 
ever, there was no additional funding for the extended 
year though the grant was entirely expended during 
that time. 

III. PHYSICAL SETUP FOR THE PROJECT 

With the NSF grant, Skidmore purchased the 
SRS lOOOC equipment from the General Electric 
Company. To install the System, Skidmore con- 
structed a 40 -seat classroom and related projection 
and terminal rooms as paxi of the facilities of a 
newly established Computer Applications Center on 
the top floor of the library building on the New 



Campus. The classroom is equipped not only with 
the SRS but also for movie, slide, and overhead pro^ 
jection as well as with a tape recorder. The Research 
and Development Center of General Electric provided 
programming and consulting services. Computing 
time was obtained, free of charge, on a time -sharing 
basis by arrangements with Griffiss Air Force Base 
in Rome, New York, which has a GE 645 computer. 
Our terminals were linked by leased telephone lines 
with the Rome computer, and one of the terminals can 
be connected through an interface with the Response 
System in the classroom. The whole system was in 
operating condition in May 1970, just as the academic 
year was drawing to a close. So real use of the 
facility could not begin in earnest until September. 
This fact was the justification for the request for 
extension of the grant period. This means the present 
evaluation study was based on our use of the System 
for one and a half academic years. 

IV. COMMON USES OF THE STUDENT RESPONSE 
.SYSTEM 

From the start we adopted the policy of going 
beyond some obvious and common uses of the SRS 
and of trying to foster the more creative aspects of 
teaching and learning through the use of the System. 
Two obvious and common uses, and our intuitive 
judgments thereof, are as follows.: 

1. The first is to ascertain the level or degree 
of immediate comprehension by students of the ma- 
terial presented by the instructor. He may ask ques- 
tions of the true -false or multiple -choice type, in 

the course of his presentation or at the end. Students' 
responses will show how successfully he has commu- 
nicated with them and whether there are points that 
need clarification. This use of the .SRS is important 
in large classes with scores or hundreds of students, 
but the need for this is perhaps not so great in small 
classes, where the instructor is in intimate rapport 
with his students. Nevertheless, this use is important 
in courses with a great deal of highly objective content, 
as in science and mathematics. If the questions go 
beyond asking the students to recall information and 
provoke thinking and discussion, then this use of the 
SRS is important in all academic disciplines and in 
classes of any size. It tends to encourage a more 
active learning process than a straight lecture-test 
technique would. 

2. Another obvious use of the SRS is to admin- 
ister multiple -choice or true-false tests to large 
numbers of students with minimum labor and maximum 
speed in scoring and analysis of results. If this is 

for the purpose of grading students in courses, we 
are rather critical of this use. In view of the present- 
day thinking ,.nd attitudes of. students regarding their 
own education, we do not want them to feel that here 
is a mechanical system to manage their learning and 
to induce in them conformity in thinking as the average 
objective tests arc wont to do. Testing for experi- 
mental purposes, with the cooperation of students and 
not for grading would, of course, be a different matter. 
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V. USES OP THE STUDENT RESPONSE STYSTEM 
AT SKIDjMORE 

Following are some of the actual uses of the SRS 
made by a number of the Skidmore faculty. The 
selected examples do not exhaust the "repertoire" 
of any single instructor or the potentiality of the 
System, but they aim at a variety in uses« It will 
be observed that none of the ways used the System 
for grading students and all of them represented 
attempts to use the SRS to promote active teaching- 
learning processes, wh?ch departed in varying degrees 
from the simple pattern of straight lectures followed 
by tests of retention. Many used the student re- 
sponses as a point of departure for teaching. This 
tended to increase students' feeling, of relevancy of 
what was taught to their own interests and needs. 

The sample uses are grouped under four cate- 
gories according to four different ways of using the 
SRS, for each of which there is a distinctive computer 
analysis program. These programs except one were 
furnished by GE. 



A. 



Using the SRS to Teach Understanding, K nowledge^ 
and Skills by Asking Questions Having Correct "~ 
Answers 



1. Sentence variations to teach punctuation and 
stylistics . In an English class for culturally de- 
prived high school graduates preparing them for 
college admission, an instructor used the Response 
System in one period to teach punctuation, specifically 
the correct uses of the comma and the semicolon. 
He projected on a screen a series of sentences, each 
in several variations, involving the use of the two 
punctuation marks. Students were asked to pick out 
the correct version in each case. Discussion of 
these cases constituted instruction. This was an 
example of inductive teaching of a grammatical rule. 
Computer analysis of student responses showed how 
each student did, as well as which questions were 
difficult or easy, thus providing a guide for subse- 
quent instruction. (See Fig. 4,) 

The same technique was used in a class in Fresh- 
man Composition to teach stylistics. A number of 
short paragraphs were shown to the class by over- 
head projection. In each paragraph, after several 
sentences of context material, a "punch-line" sen- 
tence was shown in four variant forms, all grammat- 
ically correct. The students were asked to indicate 
their preferred form by pressing the appropriate 
button, and their choices were immediately reflected 
in the percentage meters on the instructor's panel. 
They were asked to explain their choices, which re- 
sulted in a lively discussion. Strictly speaking, in 
this case of stylistics there is no correct choice, 
although the instructor indicated his own preference 
for one form, or alternative forms, depending on the 
purposes of the writer. 
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Output of Computer Pro gram "ANAL 1*^ Annly^in^ 
Responses to Questions Having Correct Answers. 
"Discrimination Index" indicates the power of a 
given question to distinguish between the top quarter 
and the bottom quarter of the class. The higher the 
index, the greater the power. The conversion of 
raw score into "T-scofe" makes possible comparison 
of student performance on different tests, if "normal 
curve" distribution of abilities is as.sumed. 



Figure 4. ANAL 1 OUTPUT 

2. Diagnosis of a class or of individual students . 
A professor in Foundations of Education used the 
SRS to administer a true-false test in order to ascer- 
tain what the students already knew or did not know 
as regards the unit she was about to teach. This 
information enabled her to concentrate on teaching 
those things which the students did not know. It 
also served to motivate the students as they could 
see clearly what they needed tq learn and thus guided 
their readings in assigned references. 

A professor of English used the SRS to administer 
a diagnostic test in English grammar consisting of 
50 questions of the objective type. The printout on the 
computer terminal immediately available upon com- 
pletion of the test as well as the later analysis by 
the computer showed clearly the strengths and weak- 
nesses of each individual student without tedious 
analysis performed manually by the instructor. Re- 
medial instruction individually designed for each 
student could then be given> 
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3. Other examples under this category of use 
wore: (a) A Chemistry professor used the SRS to 
test student understanding of a concept and its impli- 
cations, some of which he did not discuss but expected 
his students to deduce, if they had correctly under- 
stood the concept, (b) A mathematics professor asked 
students to respond to several questions at the begin- 
ning of each period, highlighting the major points of 
the last lecture and serving as a springboard for the 
current lecture, (c) A professor of Psychology used 
the SRS to teach the relation of standard deviation to 
normal frequency distribution in a variety of statis- 
tical operations, (d) A biology instructor conducted 
near the end of a semester a quick review session 
during which students responded through the SRS to 
a long list of objective-type questions covering the 
major areas of the course, thus revealing to each 
student his own weaknesses in advance of the real 
semester examination. 

B. Using the SRS to Conduct an Opinion Poll or to 
Express Preferences 

Naturally, under this category of use, questions 
have no standard answers. 

1- Studying contemporaiy'sHiTent va^ 
'^ores . A class of 30 students in a course on Educa- 
tion and Culture answered the questions dealing with 
present-day student values and mores in a national 
survey by LIFE Magazine (Vol. 70, No. 1, Jan. 8, 
1971, pp. 22-30) and compared their own results 
with those of a national sample. Since the questions 
were answered by "Yes" or "No," computer analysis 
simply gave the percentage of students choosing option 
§1 (Yes) and that for option ii2 (No) under each ques- 
tion. (See Fig. 5.) The results served as a basis for 
discussing education in relation to culture. 

2. Managing discussion in committee meetings. 
Student organizations also used the SRS. A student 
committee of 23 members conducted several meetings 
in the Response System classroom discussing what 
students wanted to recommend to the faculty in regard 
to certain all-college requirements, such as a foreign 
language. Freshman English, and physical education. 
The chairman presented a question dealing with a 
specific requirement with four alternative ways of 
handling it plus a fifth option for a "way not covered 
by the preceding four, " as follows; 

(1) Maintenance of the requirement. 

(2) Abolition of the requirement* 

(3) Reduction of the amount of the requirement. 

(4) Exemption from the requirement on demon- 
stration of proficiency. 

(5) A solution not covered by the preceding four. 

She used the SrS to find out the initial reactions of 
committee members. Then they discussed the various 
alternatives, and students voting for the fifth option 
aired their views. She took soundings again. On the 
basis of majority position she formulated a proposition 
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Output of Computer Program "OPINION^ " Analyzing 
Responses to Questions Having No Correct Answers. 
It gives the number of students choosing each option, 
and its corresponding percentage^ ander each question. 
Percentages are calculated in terms of the total num- 
ber of students responding to a given question. E. g. , 
six students failed to Answer Que.stion Hi; this number 
was ignored in calculating percentages under this 
question. 



Figure 5. "OPINION" Output. - 

and asked the committee members to express agree- 
ment or disagreement with it through the SRS. (For 
technique, see Category C following. ) She repeated 
this several times, /'ach time refining or qualifying 
the proposition until she got a consensus or a large 
affirming majority. She considered the System useful 
in both recording the process pf opinion fonnation 
and eliciting discussion in terms of the breakdown of 
opinion on a particular question. Each member of 
the committee could anonymously express his thought 
as well as vocally defend it, if he desired. Thus, 
evei7 one participated in some way during the 
meetings. 

3. Other examples of this category of use are; 

(a) A professor of Physical Education in teaching a 
course on evaluation procedures and research tech- 
niques used multiple-choice questions through the 
SRS to test student .selection of research procedures 
or formulas to be employed, to determine the sequence 
of steps in research, or to evaluate research literature. 

(b) A professor of Elementary Education asked 
students to select one of five alternative techniques 
for teaching the meanings of certain words in given 
passages in textbooks, (c) A psychology professor 
compared the attitudes towards suicide of two groups 
of students, one of which had had a "bull" session 
with him on the subject, but the other had not. (d) A 
lecturer in Asian Studies discussed with his students 
their reactions to foreign films monitored by the SRS. 
(e) An instructor in Chinese history explored student 
background for and present interest in area studies as 
well as their attitudes toward public issues relative to 
Asia, (f) A Geology professor asked his students to 
evaluate an experimental part of his course anonymously 
at the end of the term by answering a list of questions 
through the SRS. (g) An orientation program making 
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explicit the mutual expectations of the instructor and 
his students in the way of teaching-learning goals, 
course contents and. methods, evaluation of student 
performance, etc. was developed for use by faculty 
members at the beginning of a term so as to achieve 
ui;derstanding and rapport between instructor and 
students. 

^- U sing the SRS as a Rating Scale to Exp lore 
Attitu des and Feelings - 

1- Using SRS as a rating scale to study attitudes . 
A visiting lecturer- at a Child Psychology Institute. • 
conducted for a group of college teachers of Psychol- 
^ ogy. explored through the SRS the attitudes of the 
group towards the behavioral point of view in Psy- 
chology. The professor presented orally a list of 
19 statements on various implications for child de- 
velopment and education of the point of view in ques- 
tion. He asked members of the group to express their 
agreement or disagreement with each statement 
using the five buttons as a five-step scale: 

"1" representing "Total disagreement. " 

"2" representing "Disagreement.*' 

"3" representing "Neutrality. " 

"4" representing "Agreement." 

"5" representing "Total agreement." 

The percentage meters on the instructor's panel 
instantly showed him how the group reacted to a given 
statement. He announced the percentages to the 
group, followed by comments by him or by partici- 
pants. It was interesting to note that his first state- 
ment was the most general and comprehensive, and 
his 19th statement was identical to the Cirst. For 
spme participants, their responses on the 19th indi- 
cated a significant shift in attitudes from their re- 
sponses towards the first statement, apparently 
caused by their reacting to the intervening ones. 

In computer analysis of rating responses. But- 
ton jfi is arbitrarily chosen not as a standard but as 
a reference point in scoring. A students' response 
is scored in terms of the number of steps it is away 
from "1. " "Four" is the maximum score for any one 
question. (See Fig. 6. ) If Button #3 represents 
neutrality." as in the case above, then a score of 
2 means "neutrality. " The farther up a score is 
above "2," the stronger the tendency towards "total 
agreement. " On the other hand, the lower down a 
score is below "2, " the stronger the tendency towards 
total disagreement. " 

However, if a scale represents linear progression 
from zero to maximum (e.g., "l" = "Useless " 
"2" = "A bit Mseful;" "3" = "Moderately useful;" 
"4" = "Quite useful;" and "5" - "Very useful.") then 
the higher the score, the stronger the tendency to- 
wards the upper end of the scale. 
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Output of Computei- Program "ANAL 2" Analvittn ^r 
Responses on a Uatinp Scale . SrOnp ^ ^.e f,^^ 
away from "1. " A response on #3 on the scale is 
.seored as "2" (2 stops from "1"). A response on 

on the iseale is seored as "4.*' which is the maxi- 
mum seore for any on<^ question. In PUOBLKM 
ANALYSIS. TOTAL SCOUE is the sum of the scores 
of all the students on that problem, and MEAN SCORE 
is obtained by dividing TOTAL SCORE by the number 
of students. In STUDENT ANALYSIS. TOTAL SCORE 
IS the sum of the seores on all problems achieved 
by that student. anp^MEAN SCORE is obtained by 
dividing TOTAL SCORE by the number of problems 



Figure 6. ANAL 2 OUTPUT 

2- Other examples of this catef^ory of use are; 
(a) A Biology professor generated class discussion 
of an assigned reading by asking students to express 
through the SRS their agreement or disagreement with 
the general point of view of the author as well as with 
specific statements he made, (b) After using the 
SRS in the teaching of a class or course, the instructor 
asked the students to rate on a five-point scale the 
usefulness of the Response System in that particular 
class or course, (c) A group of students interested 
in admission procedures of the College was asked to 
express the degree of agreement or disagreement 
with the hypothetical proposition that Skidmore hence- 
forth cease using SAT-Verbal and SAT-Math scores 
in considering an applicant for admission. 

Using the SRS to Establish Priorities 
*» 

Studying priorities in value systems of dif- 
ferent societies. A professor of Asian studies asked 
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his students, who happened to be all women, each to 
give him a list of several fact«.r« or values she would 
look for in selecting a prospective mate. From these 
lists he made a compofjite list of 22 factors. The 
first step was to find the five top values for the class 
as a whole. The students were asked to examine the 
list of 22 factors and each to select five factors most 
important for herself. As each factor was announced 
by the instructor, each student pressed either Button 
#5 to indicate it was among her top five values or But- 
ton #1 to say it was not, but never used the in-between 
buttons. From the immediately available printout of 
raw data, the injstructor could readily select the five 
factors with the most "S's" and the least "i»s." The 
second step was to establish an order of priorities 
among the top five. Students were asked to assign 
mentally one of the five buttons to each of the five 
factors, with no button used more than once and with 
' 5" representing the highest priority and "1" the low- 
est among the top five. Then, as each factor was 
announced, students pressed the appropriate button, 
(For computer calculation of weighted scores, see 
Fig. 7. ) The results were: first, love; second, 
intelligence and education; third, compatibility; fourth, 
integrity; and fifth, sense of humor. 



W«»ttrn MftrrUcei 

CompitibfHty 
Itumor 

Inttll.Xr Cduc. 
Love 

He*Jtb k r«rUI. 

Lov« of Children 
Comp. r*m. Bkf rd. 
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Output of Computer Program "TXSCALE/^ Estab- 
lishing Priorities Among Items . "Weighted Scorp'^ is 
obtained by multiplying each button value by the num- 
ber of persons choosing that button and then summing 
up the products. E. g. , for "Compatibility" it is 
(0X0) + (ixi) 4- (5x2) + (5X3) + (6x4) + (1x5) ■« 55. 

Mean Score" is obtained by dividing 55 by the num- 
ber of students participating (18). Since Button 15 
was used to represent the highest priority, the larger 
Mean Score, the higher the priority. 

Figure 7, TXSCALE OUTPUT 

Next he reminded the class of its earlier stuc^y of 
the traditional family system in China, under which 
marriage was no* a personal but a family affair, and 
as such, it was f.rranged by parents with the primary 
objective of producing offspring. The class was 
asked to suggest a list of values Chinese parents 
would look for in a prospective daughter-in-law* 
Again, through the use of SRS, the top five values and 
their priorities were ascertained. They were: first, 
health and fertility; second, comparable family back- 
ground; third, humility and obedience to parents-in- 
law; fourth, loyalty to family; and fifth, love of child- 
ren. It became obvious to the class that traditional 



Chinese and contemporary Western marriages had 
very different value systems. None of the top five 
Chinese values was found in the composite list of 22 
Western values. 

It was pointed out to the class that marriage was 
merely a single illustration of a general principle 
very important in the study of foreign cultures and 
societies. What is this principle? Class discussion 
concluded with this generalization: each society or 
culture has its own value system appropriate for each 
of its institutions; you cannot impose the value system 
of one society upon another unless you change the latter 
society. The lesson was a good example of having 
students perform a demonstration of an abstract 
principle. The SRS with its capability of instant feed- 
back made it possible to compress the whole teaching- 
learning process into a 50-minute period, with no 
tedious calculations on the part of the instructor or 
students. 

2« Othei* examples of this category of use were; 

(a) In a class in Dance Appreciation, students were 
shown a film of two Chinese, two Korean, and one 
Japanese dance and asked to arrange the five dances 
in descending order oi personal likes and explain why, 

(b) A group of college admissions officers arranged 
in order of priorities five specific factors governing ^ 
the admission of applicants. ^ 

VI. COMBINED USES OF THE STUDENT RESPONSE 
SYSTEM IN CLASS 

To recapitulate, the above illustrative uses of 
the SRS are classified into four categories according 
to the nature of the questions used: 

A, Using the SRS to teach understanding, know- 
ledge, and skills by asking questions having 
Correct answers, 

B, Using the SRS to conduct an opinion poll or to 
express preferences by asking questions 
having no standard answers, 

C, Using the SRS as a rating scale to explore 
attitudes and feelings, 

D, Using the SRS to establish priorities. 

In any one instructional period, the instructor may 
use any combination of categories to suit his teaching 
objective. However, if he wants computer analysis of 
student responses to be done after class, then the 
records of responses to different kinds of questions 
must be separated before transmission to the com- 
puter, because each type of question calls for a 
different computer analysis program, (See Figs. 4 
through 7,) 

VII, USES OF COMPUTER ANALYSES 

For many faculty users of the SRS, the information 
gained from the percentage meters on the instructor's 
panel was sufficient for leedback purposes. The in- 
structor would not know which student responded in 
what way, but he could readily see what percentage 



of the class reactotl in what way. This Is the 'easiest 
and the commonest way of using the Response r^^ctcm. 
Since this information is instantaneously avaiLVole, it 
tends to affect the teaching-learning process in medi- 
ately. This use is important both in teaching basic 
knowledge and in directing discussion. 

Some instructors wanted, in addition to informa- 
tion from the perc-finta^je meters, a recording of 
student responses /n the form of a printout on the 
teletypewriter ava<<&bln t'^lmoft immediately. It takes 
only five seconds t(» record %) d print out the responses 
of 40 students as w«U Jip thM 'yt the instructor to one 
question. This information is more exact than the 
readings of percentage meters since one can tell from 
the printout how the student at each occupied sc^at 
responded. (See Fig. 3.) If the printout is posted 
after class with a list of the questions used and if a 
student remembers wliere he sat, he can compare 
his own responses with those of th^ instructor and 
those of his fellow students. 

Other instructors definitely wanted computer 
analysis and regarded the step of recording as in- 
cidental to the step of creating a paper tape with the 
responses encoded on it, by means of which the data 
could be transmitted through a telephone wire to a 
distant computer for analysis according to the pro- 
gram selected. Since we had access to the computer 
in Rome, New York, only in ♦he afternoon of a week- 
day, the analysis could be made available to the in- 
structor on the day of his class or in the following 
morning, depending on the hour of his class. ^ 

Since the wishes of the faculty users varied in 
the above way, we found it useful to organize our 
written instructions on "How to Operate the Response 
^stem" into three parts: Part I, How to Operate 
the Percentage Meters; Part II, How to Obtain a Re- 
cording; and Part III, What to Do to get a Computer 
Analysis. Part I was included in Part II, and Parts 
I and 11 were included in Part III. Each user could 
pursue the instructions only as far as his interest 
went. As a user gained confidence in using the Sys- 
tem in a simple way, he could move on to a more 
complex way. This avoided the danger of "frighten- 
ing*^ or confusing a new user with a lot of complicated 
instructions given in one shot. In briefing faculty 
members, we had to emphasize frequently how easy 
it was to operate the System. 

As to computer analysis, there were four pro- 
grams to t9ke care of the four major uses of the Re- 
sponse System as outlined in the preceding two sec- 
tions and a fifth one for changing the format of data» 
There were other programs to treat data collected 
by the System in the Free Mode. (See section follow- 
ing.) All programs, except one, for analysis of SRS 
data were created and furnished to us by General 
Electric. The exception, called "OPINION, " was 
created by our own staff. Following are some com- 
mon uses of various programs: 

1. ANAL I : This program scored student re- 
sponses according to a standard key supplied by the 



instructor or conect answers entered by him Uuough 
the instructor's panel. (Sec Fig. 4.) Its greatest 
usefulness was in connection with giving tests or 
quisszes, in which every question had one and only 
one correct answer, whether for the purpose of gen- 
erating discussion or review or for grading students, 
ll yielded statistical information on the performance 
of the class as a whole as well as that of individual 
students. A feature of the program not illustrated in 
Fig. 4 was the capability to make individual diagnosis 
by instructing the computer to conduct as many "passes 
over the data as there were sub-areas of the test, 
yielding for each student a si.b-score for eaclj ari'U 
as well as a total score for the whole test. 

ANAL 1 was also found vei*y useful for item anal- 
ysis of a test. The section of the output called "Prob- 
lem Analysis" showed what percentage of the class 
had answered each question correctly, and from the 
"Response Count" for each question the Instructor 
could see not only the number of students getting 
the correct answer but also the most "popular" 
errors. Such item analysis enabled the instructor 
to improve his test and his technique of testing. 

2. OPINION : This program was very useful in 
analyzing student opinions on questions and issues 
having no standard answers. It simply gave the num- 
ber and the percentage of students choosing each 
Option under each question and left it to the instructor 
and students to interpret the significance. (See 
Fig. S. ) Though the Response Count in ANAL 1 gave 
the number of students choosing each option under 
each question and so could also be used to study the 
distribution of student opinions, many instructors 
balked at the use of ANAL 1 for this purpose, be- 
CTiuse the program presumed one correct answer for 
each question. So OPINION v/as created to tabulate 
opinions from a neutral or open-minded point of view. 
It also had the advantage of giving both numbers and 
percentages of students under each option. 

A new use of this program has been planned. The 
author of a new text in General Psychology will let a 
class use the book in manuscript form. Every major 
paragraph will be numbered. At the beginning of 
each class period, the students will be asked to spend 
a few minutes evaluating each numbered paragraph 
in the chapter assigned for that class period by 
answering a question such as the following: 

"How would you characterize this paragraph? 

*'(!) Meaning not clear 

"(2) A concrete example would help 

"(3) Language too involved or difficult 

"(4) Content uninteresting or not relevant 

"(5) Satisfactory" 

On the basis of student comments the author will re- 
vise the manuscript. 

OPINION wan tailored for data on opinions entered 
through the SRS in a class situation. For such data 
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entered through the Response System in the Free 
Mode (see next section), there was a program, 
called "NLSCORE, " which accomplished the same 
purpose., 

3. ANAL 2 > This program analyzed student re- 
sponses when the five buttons of the Input Station were 
used as a hve-step sc^le for ra ting purposes. (See 
Fig. 6.) It was found very useful in all sorts of 
student evaluations of instructional materials, courses, 
and instructors. It was also frequently usetl to stim* 
ulate class discussion by allowing students to express 
candidly their agreement or disagreement with what 
was said by the instructor or the textbook, etc. The 
SRS encouraged a 100% student participation. 

4. TXSC.^LE; This program yielded weighted 
scores for various items to be arranged in order of 
priority in the judgment of a class. (See Fig. 7. ) 
Since priorities are usually subjective, this was 
another way of studying student opinions. 

5. LSTOMAN : This program was devised to 
surmount the limitation of ANAL I and ANAL 2 to 
responses from a single class or section of students 
because of the identification of students by seat 
numbers.. When students from two or more sections 
of the same course responded to the same questions 
we used LSTOMAN to combine the data into manual 
form, inserting section numbers to differentiate 
students who occupied the same seats but in different 
sections. The combined data could then be processed 
by one of the ANAL programs. 

In summary, the main point to be emphasized in 
discussing the use of computer analyses is that anal- 
ysis by computer saved the instructor a lot of time 
and from a great deal of drudgery; and hence, tended 
to encourage. him to obtain such analyses more fre- 
quently than otherwise, thus hopefully resulting in 
improvementrof instruction. ^ — ^ _ — 

VIII. INDIVIDUALIZED USES OF THE RESPONSE 
SYSTEM 

The above ways of usinf the SRS illustrate its 
operation in the Group Mode, i.e. , in class instruc- 
tion. The System can also operate in the Free 
Mode, which facilitates individualized instruction 
and independent study. Students working with in- 
dividualized instructional material, whether "pro- 
grammed" or otherwise, may come to the Response 
System classroom at any time and enter their re- 
sponses to questions through the SRS. Different 
students working on material in different courses may 
work at the same time in the classroom, each at 
his own pace. Their responses will be continuously 
scanned and recorded on a teletypewriter in mixed -up 
order. ID numbers for students and courses will 
necessarily be used so that the computer may sort 
out the student responses by courses, by students in 
each course, and by questions under each student's 
ID. After sorting, the computer will score the re- 
sponses and analyze the results in a manner specified 



by the analysis program selected. To test this 
capability of the SRS, a few examinations in Biology, 
Psychology, etc. were administered through the 
System operating in the Free Mode. At first, we 
experienced great difficulty in having Free Mode data 
processed by the computer we regularly used, but 
when we shifted to the computer of the GE Research 
and Development Center, for the processing of such 
data, the results were very satisfactory. 

During the spring semester of 1970-71 the 
Department of English ran an experiment to compare 
the effectiveness of three different approaches to the 
teaching of Freshman Composition. It involved 11 
sections of this course, with a total enrolment of 
191 students. The results of the pre-test and post- 
test were processea oy the combined use of SRS and 
the computer. 

The SRS is also a convenience to students or 
faculty doing independent research projects involving 
quantitative studies. For example, questionnaire 
returns can be entered through the SRS, provided 
there are no more than five options under each ques- 
tion and it is understood that any open-ended ques- 
tions with written-in answers will have to be scored 
by hand. There ic a computer analysis program which 
will give, on a ^irintout, the number of respondents 
choosing each option under each question and its 
corresponding percentage in terms of the total num- 
ber of people answering that question. This particu- 
lar analysis program called "NLSCORE, " can handle 
questionnaires with up to 99 questions and up to 199 
respondents. " 

Several faculty committee questionnaires con- 
cerning curriculum questions, ^one involving as many 
as 600 returns, were processed in the above way. 
This saved the committees a lot of tedious work 
otherwise involved in manual calculation. Again, 
— more-than a dozen questionniare studies conducted 
as individual projects by students in,Business and 
Nursing were processed, with the students learning 
the techniques of processing. 

There are, of course, other forms of quantitative 
studies compatible with treatment by the SRS. The 
point to be emphasized here is that the researcher, 
whether student or faculty, can be spared from time- 
consuming and tedious tabulations and calculations 
and so focus his main attention on analyzing his^ 
research problem, formulating his design, collecting 
his data, thinkir^ through the procedure of treating 
his data, and when the results of analysis are delivered 
by the computer, interpreting the results and drawing 
his conclusions— precisely those steps in his project 
that are most educational and least mechanical. The 
time required for completion of work is speeded up 
immeasurably by the computer. 

Plans have been laid to encourage faculty members 
to produce material for individualized instruction 
either in connection with their courses or as indepen- 
dent study projects. A group of ten professors from 
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as many academic disciplines has agreed to author 
instructional material* accompanied by appropriate 
media* each from his own disciplinary point of view 
but all focused on a common theme. Students will 
be encouraged to achieve some degree of mtegration 
based on their exposure to multidisciplinary materi- 
als. The Response System wilt probably be of use 
in monitoring student responses as feedback infor- 
mation to the student to help him improve his learning 
and to the faculty writers to enable them to improve 
th#'ir instructional materials. But all this is in the 
future 

IX. EXTENT OF USAGE 

Just how much use did the Student Response 
System get and in what kinds of courses? Courses 
were scheduled to meet in the SRS classroom at the 
request of the instructors concerned. In order to 
invite faculty requests* general announcements were 
made to the entire faculty, at least once a semester, 
urging them to inspect the facility and decide for 
themselves whether they could usefully employ it in 
their courses. Surprisingly, such general announce- 
ments brought forth little interest. This fact seemed 
a very interesting commentaiy oa the present general 
feeling among liberal arts college teachers towards 
machines and technology. 

Accordingly, a second approach was employed. 
"Hopeful prospects" in the faculty were individually 
invited to come for a personal briefing in regard to 
the operation of the System and its potential uses in 
undergraduate teaching. Each decided for himself 
whether he would like to try it. In the one and a half 
years the SRS has been operating, the number of 
faculty members thus briefed exceeded 70, which 
amounted to nearly 50% of the whole faculty. Re- 
cently, two academic departments requested briefings 
and di^monsrrations for their departmental staff. 
Such leadership tends to encourage faculty use; 

As a result of these briefmgs, 29 different faculty 
members (about 40% of those briefed) each decided 
to and did hold one of his classes in the SRS class- 
room. They taught in this room a total of 35 different 
courses taught to 45 different classes, distributed 
as follows; 



Courses Classes 



Mathematics, Biology, 
Chemistry, Geology 

Psychology 

Economics, Government, 
Sociology, History 

English, Asian Studies, 
Dance, Art 

Business, Elementary . 
Education, Physical Educ* 

TOTAL 



11 
5 

5 

8 

_6 

35 



16 
6 

5 

11 

J7 
45 



The average enrolment in these 45 classes is 
over 15 students each. This would mean no less than 
675 students, representing about 37% of the total stu- 
dent body, have participated in the use of the SRS in 
the last one and a half academic years. 

The extent of use among these 45 classes varied 
widely, ranging from several class periods through 
a few weeks to steady use throughout the semester. 
Another way to indicate the amount of use is that 
during the first academic year the average number 
of class periods per week, in which classes were 
conducted in the SRS classroom, was 10. averaging — 
two classes per week day. During the fall semester 
and Vv'iuZcr term of the second year, the average in- 
creased to over 15 class periods per week or 3 per 
week day. representing a 50% increase in use. 

Of the 29 faculty users of the SRS, thirteen were 
"steady" users in the sense that all meetings of tlieir 
classes took place in the special classroom throughout 
the duration of the course. The other sixteen were 
occasional or short-term users. Of the total num- 
ber of faculty i««;ers (29). fourteen were "repeaters" 
in the sense that, after their first experience with 
the SRS. they either taught another course using it 
or the same course to a different class. 

Some of the "non-repeaters" fir jt tried out the 
System in the fall of the second year, and at the time 
of the writing of this report, they had not yet had an 
opportunity to repeat. A few had the opportunity^ but 
did not choose to repeat. 

When all the data on use are considered, we can- 
not say that the SRS took to instant popularity. Yet 
the GE representative cooperating with Skid more in 
the project told us Uiat we weri? doing "fantastically" 
well, considering the size of our institution as well 
as the fact that we relied chieily on voluntary efforts 
of faculty members in developing instructional ma- - 
tcrial for SRS teaching.* 

The reasons for a relatively small start are: 

(1) college faculty anywhere is usually cautious in 
adopting new changes in curriculum and methods; 

(2) many instructors are more subject-matter- 
centered than student -centered in their approach to 
the problem of college teaching; (3) some instructors 
are far more concerned with the highly sophisticated 
outcomes of humanistic learning which the Response 
System cannot bring out but require the writing of 
essays, or constructed answers to questions, or 
other complex modes of express icu; and (4) Skidmore 
is in transition from its old campus to a gradually 
built-up new campus. During the period under review 
about two -thirds of its classes; were held on the New 
Campus, while the remainder was still conducted on 



^For a more detailed discussion of this point, see 
Question 8 under "X. Evaluation of the Response 
System. " 
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the old with a class schedule half an hour behind that 
of tlie New Campus in order to permit students to 
commute between the two locations. As a consequence^ 
classes scheduled for the old can>pus couUi not have 
short-term use of the SRS cIa:>sroom on the Xew 
Campus. (5) The capacity (*f this special classroom 
being 40 scats precluded very large classes from 
using the classroom^ even if the inst motors had so 
desired. * 

X. EVALUATION' OF THE RESPONSE SYSTEM 

In our original application to the National Science 
Foundation, it was stated in reference to evaluation: 
"Evaluation should aim primarily at finding out how 
many different ways tlie System can be used to pro- 
mote 'creative' teaching, in what academic disciplines, 
and for wliat educational objectives. ... I:^'/aluations 
sliould be specific and relevant to each way it (tlie 
System) is used. ...the instructor and his students, 
should rate the System to indicate the extent to whicli 
the facility has been Iielpful in acliieving tlieir educa- 
tional objectives — However, it should be clearly 
understood that the evaluation. . . will be mainly 
qualitative. Statistics will receive a secondary 
emphasis. ..." 

In accord with the above statement, our evaluation 
will be qualitative and analytical ratlier than statisti- 
cal. Tliree sources of information will serve as the 
.basis for evaluation: (a) data on usage; (b) faculty 
and student ratings; and (c) opinions of faculty users 
obtained by individual interviews and in group dis- 
cussion. The greatest ini{)ortancc is attaclied to the 
third source, for there we try to go beltind and be- 
yond numerical figures and to interpret our total 
CNperience with tlie Response System. We shall 
organize our observations around certain questions, 
frequently raised by visitors, and around the evalu- 
ative remarks made by our faculty users. 

I. Is the SRS Better Fitted for tlie Teach ing^of Some 
Academic Disciplines Tlian Others? 

Our data on usage sliowed that tlte SRS w^s used 
in 35 different courses belongingto 15 different academic 
disciplines representing the Immanities, mathematics 
and sciences, social sciences, and professional fields. 
The question seems to imply tliat tlie Rcsoonse System 
permitting student response only to objective -type 
questions is perliaps better suited to the teaching of 
the so-cailed highly objective disciplines such as 
mathematics and the -sciences. But tltis view over- 
looks the {K)ssi|)ility of using Uie System to generate 
class discussion of opinion- issues in the less exact 
disciplines.- Also,— the nature of any academic dis- 
cipline and the nature of the teaching -learning process 
are both so C9mplex that tliis dichotomy into "objec- 
tive" and "subjective" disciplines is false. Every 



discipline has some objective content and some sub- 
jective elements. It is our considered opinion that 
theoretically any instructor in any discipUnr, possibly 
witli the exception of courses primarily concerned 
with t!*e development of performance skills, could 
find some use for the SRS in certain aspects of his 
teaching, if he chose to do so. Faculty users are 
among tliose who are aware of the importance of tlie 
role of student responses in tlie teacliing- learning 
process. So the extent of use varied with the in- 
structor rather tlian with tlie discipline. 

2. , Is tlie SRS Useful Primarily for the Purpose of 

Checking Student Understanding of Lectures and 
iVdministering Objective Examinations? 

As detailed in tlie section on "Uses of the Student 
Response System at Skid more" above, our faculty 
users have emphasized a variety of ways of using the 
System to encourage a more active participation ii» 
the learning process on the part of the students and 
a greater adaptation of ilie teaching process to stuuent 
interests and needs on the part of the instructor. The 
uses departed in varying degrees from the usual 
lecture-test pattern of instruction. 

3. In the Opinion of the Faculty Users and the 

. Students in Their Classes, was the Response 
System Useful for the Purposes They Had in 
Mind? 

We requested every instructor to ask Itis stu- 
dents, at the end of a period of use, to answer anon- 
ymously the following question through the SRS and 
that he himself also answered it; 

"How useful has the Response System been 
to the teaohing- learning process as con- 
ducted in this classroom in this course? 
Rate its usefulness according to tlie 
following scale: 



Useless 



2 ~ 
A bit 
useful 



Moderately 
useful 



4 

Quite 
useful 



5 

Very 
useful' 



-Very small classes with less than 10 students each 
would present no problem as the System could be 
specially calibrated to take care of small numbers. 



Our records are not complete since some in- 
structors did not liave time or did not choose to 
participate in the evaluation. Nevcrtlteless, ratings 
were obtained from over 64% of the classes. We 
shall resist the temptation to give an average num- 
erical figure as a statistically precise index of use- 
fulness. Since the purpose for which the System was 
used, the duration of time in its use, and the number 
of students involved differed so widely that any average 
numerical index would be as meaningful as adding 
apples, oranges, and grapes together and takfng an 
average of them. For what they are worth, we may 
summarize faculty and student ratings of the SRS for 
their own purposes as follows: 

(a) Faculty ratings turned in range from "2" 
(A bit useful) to "5" (Very useful), with the majority 
for "3" (Moderately useful), and 40% for "4" (Quite 
useful) and "5" combined. A very few faculty users 
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known to be critical of the SRS did not turn in ratings 
for themselves or their students. Had they done so* 
"1" (Useless) might have been their rating. 

(b) As for student ratings* individual ratings 
ranged from "1" to "5* ** with the great majority for 
"3" (Moderately useful). Average student rating was 
computed for each reporting class. These class 
averages ranged from "2. 1" to "4. 7" with the median 
average rating at "3. 3. ** 

(c) la two-thirds of the classes ivith ratings* 
the instructors* ratings were somewhat higher than 
the students* average ratings. In only one class was 
the instructor's rating the same as the average stu- 
dent rating for that class. 

4. In Teaching, What Can Be Pone Through the SRS 
That Cannot be Achieved Without It? 

The Respon.ee System is uniquely useful: 

(a) when anonymity of student response is es- 
sential. For example* in collecting information re- 
garding drug use or sex practice among students* the 
instructor can assure them of anonymity by using 
only the percentage meters on the instructor's panel 
and by not recording. Seats are far enough apart and 
the buttons in the Input Station are close enough that 
even neighliors cannot easily observe how a student 
is responding. If he shields the Station with one 
hand while responding* he can be absolutely sure of 
privacy. If the instructor desires recording and 
subsequent computer analysis of data* then he should 
allow students to sit anywhere they like and promise 
not to take any special notice of who sits where. 
This use of the SRS is better than asking for written 
reports without a signature* because handwriting can 
sometimes be recognized. Anonymity also encourages 
students to express freely views that are apt to be 
unpopular with their classmates or unfavorable to 

the instructor; 

(b) when 100% student participation in discussion 
is desired. Some feel that the use of the SRS* while 
justifiable by the instant feedback not otherwise avail- 
able in a very large class* is really unnecessary in a 
small one. It is true that if an instructor maintains 
good rapport with a ^mall class* there can be a free 
flow of dialogue between them. Yet, in such a situa- 
tion very often a few bright or aggressive students 
tend to monopolize the dialogue with the instructor* 
the rest of the class just sitting back as silent 
spectators. With the use of the SilS* every student 
has to be active to the extent of^making a decision 

on the question raised. After the responses are col- 
lected and noted* the instructor may indeed throw 
open the question for free discussion. Having com- 
mitted themselves to a certain decision* even the 
formerly "silent" students would be more inclined to 
speak up for their positions. Hence* class discussion 
will be more widespread and the learning process 
will be more active for all students; 



(c) Wlien speedy recording of responses and 
quick computer analysis; are desired. Needless to 
say* this use saves the instructor from a lot of 
tedious work and therefore will probably encourage 
him to take more frequent soundings of student re-> 
sponses. 

5. I>?esn*t the Operation of the Response System 
Tend to "Intervene" Between tlie InstHictor and 
His Students? 

.^fter asking a question* the instructor has to 
push two or three buttons and the students have to 
press one before he can talk about their responses. 
This intervention is disliked by many students and 
is distracting to the instructor. His train of thought 
is interrupted by his operation of the machine. This 
negative feeling is so strong in the case of a few 
faculty users that they would not use the System 
again. There is no good solution to this problem. 
Each faculty user has to decide for himself the 
question of trading off this undesirable feature for the 
advantages of the System. 

Recently, two faculty users who didn't care to 
handle machinery had one or two students in their 
classes trained in the operation of the SRS. At the 
beginning of each class period* a student would make 
ready the System. The instructor might on the spur 
of the moment use the SRS* and the student would 
operate it for him or the instructor might never use 
it in that period. It does not hurt the equipment to be 
in readiness for any length of time. This solution 
tended to reduce the feelihg of distraction. 

6. Are there any Inherent Weaknesses in the 
Response System? 

One weakness is that it cannot accept a con- 
structed answer to an open-ended question. Nothing 
can be don. to overcome this. Another limitation is 
that the number of options under any question cannot 
exceed five* though it may be less than five, as in a 
true-false question. A partial solution is to use the 
fifth option of ''Other alternatives. " When an in- 
structor sees a significant percentage of students 
choose the fifth option, he should ask students to air 
their own ideas. Another solution is to repeat a 
question* each time having up to five choices. Stu- 
dents who find their answers under the first appearance 
of the question will ignore its second appearance* or 
vice versa. 

7. Shouldn't Students Have the Educational Experience 
of Asking Questions and Discovering Many Dif- 
ferent Ways of Answering a Question Instead of 
Having Questions and Multiple-Choice Answers 
Provided by the Teacher All the Time? 

In the experience of some instructors, the 
multiple-choice answers under a question often sharp- 
ened the student perception of the question* and in the 
ensuing discussion the students frequently gave more 
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diver.se responses than when a question was asked in 
the traditional way. 

However, certainly the instructor should not be 
the one to ask questions and provide alternative an- 
swers all the time. In a free-wheeling discussion, 
the questions as well as the options under each ques^ 
tion may indeed be suggested by the students them- 
selves. They may be written on tiie blackboard, 
obviating previous preparation, and then be responded 
to by the students througli the SRS. For example, in 
a Dance Appreciation class, after seeing two films 
on Asian dances students v/ere asked to suggest im- 
portant characteristics of dances in Asia insofar as 
they wore illustrated in the films. The instructor 
listed their suggestions on the blackboard and then 
asked the students each to indicate through the SHS 
which characteristics were within the most important 
five in his own opinion, uninfluenced by others in the 
class. The results stimulated a lively discussion, 
in which the instructor helped the students distinguish 
the trivial from the essei^tial. 

8 Doesn't Pre-Programming of Questions Take A 
Lot of Time? 

Yes, but hopefully, some of the material de- 
veloped may be of use again when the course is taught 
next time. The more creative uses of the Response 
System do require much more of the instructor in 
time and effort--defining teaching goals, planning 
of teaching procedures, pre-programming of ques- 
tions, preparing media, etc. While preplanning 
tends, in itself, to improve instruction, quite apart 
from the use of any physical facility, many faculty 
members feel under pressure of time, that they can- 
not afford the extra time and effort involved. Ulti- 
mately, the question boils down to one of priorities 
in the use of instructor time. What does the instruc- 
tor aim to accomplish in teaching? Does the use of 
SRS to monitor promptly student responses tend to 
help him realize his goal?-Is~il worth thc-cxtra time 
and effort involved? 

One way to solve this difficult problem is to get 
expert college teachers to develop software involving 
the use of SRS in various disciplines.. Just as the 
use of computers in undergraduate instruction did not 
get very far unti? software was created in various 
subjects of the curriculum, the same is true of the 
use of the Response System. Granted that college 
teachers are usually very reluctant to adopt whole 
courses worked out by others, it does not preclude 
the possibility of producing instruqtional materials in 
the form of a large number of smalLmodular units 
on specific topics commonly found in a course for 
undergraduates, and thus allowing the instructor to 
choose whatever units he likes to be incorporated 
into his own course. A good example of this is the 
very large number of audio-tutorial units in Biology 
developed and published by the Burgess Publishing 
Company, except that these units are intended for 
individual study whereas we are talking about units ' 
for class instruction through the SRS. 



An instructor in Physical Education at Skidmore 
plans to write a unit on First Aid to be taught entirely 
through the Response System in a workshop. She 
thinks that teaching First Aid in th3 conventional way 
may be a bit boring, but teaching it through the SRS 
will point up areas that need discussion and will re- 
sult in more efficient learning in less time. While 
she is willing to make this extra effort on a voluntary 
basis, to get a large number of teachers to develop 
software in various disciplines funds are needed to 
make possible the giving of "released time" to them 
or to compensate them for the use of their free time 
in the summer. Any rapid expansion of the use of 
SRS would have to await this development. 

Administrators must not think that once the 
equipment is bought, use of it in teaching will follow. 
Resources are needed for continuous faculty develop- 
ment, if continuous improvement of instruction is 
the goal. One of the most important lessons we 
have learned from our project is that we did not plan 
adequately for faculty developjnent. While we were 
highly gratified by so many faculty members making 
entirely voluntary efforts in developing units for SRS 
teaching, the use of the System could have been much 
wider, had there been a systematic plan for providing 
faculty incentives. 

9- Doesn't Pre-Prograniming Result in InHexibility 
in Teaching Procedure and Create in the Student 
A Feeling of Being "Restricted" by the Instructor's 
Material? 

It doesn't have to. The instructor should feel 
free to depart from a prepared list of questions and 
let the class discussion go in whatever direction that 
seems desirable. The unused questions may be used 
some other time. Or, pre -program only one or two 
questions as a starter and let the next question grow 
out of the discussion of the preceding one, thus resul- 
ting in an open-ended teaching-learning process. 

Sometimes it is wise to provide an opportunity 
for students to "let off steam'* by having such options 
as *'Cop out," "A stupid question," "I have a brilliant 
answer," etc. Oral discussion after entering re- 
sponses tiirough the SRS will, among other functions, 
satisfy students who feel the need for vocal response. 

10. Does the Use of the Response System Always 
Involve Media Preparation? 

The instructional material to which students are 
to respond may be presented by the^ projection of 
films or slides, or transparencies on an overhead 
projector, by playing tapes, or by distribution of 
mimeographed material, or by staging an activity in 
process such as a demonstration, a debate, etc. But 
none of these things is required. A simple question 
may be presented orally in class or written on the 
board. Whether an instructor is to go in for audio- 
visual or other technological aids is entirely up to 
him. 
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11. Do Students Feel at Ease in Using the Response 
System? 

We have not studied the problem, but some 
students have remarked on their feeling of ''too 
much competitiveness." Such feeling may arise in 
some students* perhaps because they can see their 
own success or failure in a much more clearly de- 
fined way than in vague self-estimates without the 
use of the SRS. Is this good or bad for learning? 
Any feeling of competitiveness may be lessened by 
providing strict anonymity and by not using the SRS 
for grading purposes. 

An instructor pointed out that long-term student 
users of the Response System might be different in 
their f;3elings from occasional users. He sensed 
that some students were a little uneasy in using the 
System at the beginning of the semester, but became 
perfectly at ease at the end of the term. This ob- 
servation was in accord with a study made by Beth 
Sulzer at Southern Illinois University, who at the end 
of a semester of Educational Psychology asked her 
class of 322 students to answer the question, "How do 
you like the student response system?" on a five- 
point scale from "Favorable" to "Unfavorable. " 
Over 50% of the class responded with ^*1" or "2."- 

12. Is there Any Special Seating Arrangement in a 
Response System Classroom? 

The seats in our classroom are arranged in five 
rows, of eight each, facing the instructor. The In- 
structor's panel is mounted on a fairly large lectern. 
We chose this arrangement because it would accom- 
modate the greatest number of seats within a limited 
space. Some of our faculty users have criticized this 
seating arrangement as being too conventional and 
the lectern as too formal. In these days of informality 
irregularity and casualness are preferred, t 

13. What is the Cost of a Student Response System 

Naturally, it will vary with the nmnber of seats 
involved, among other factors. It would not be useful 
to mention the price we paid for our system, for the 
equipment was specially manufactured for Skid more, 
not a product of assembly line production, and the 
price is several years old. Suffice it to say that the 
magnitude of the cost wa.5 on approximately the same 
order as that of an electronic language laboratory 
having the same number of seats. 



The GE Student Response System lOOOC is fairly 
rugged, and service calls have been Infrequent. 

14. Is the Uso of the Computer in Connection with 
the SRS an Economical One? 

In the use of the SRS, students at no time come 
into direct contact with the computer. The System 
collects and records student responses, and when 
these are fed through a terminal into a computer, 
the latter will analyze the data according to the 
analysis program selected. Such a program may 
yield individual analysis as well as group analysis. 
The computing involves only one terminal on line 
with a remote computer in a time-sharing system. 
As compared with CAI (Computer Assisted Instruction), 
which requires at any one time a separate terminal 
and a separate. port into the computer for every 
student, the SRS use of the computer is far more 
economical. Of course, it does not have the advan- 
tage of "immediate turn-about" that CAI has. The 
mode of computer use in connection with the SRS 
may be called, "Computer Analyzed Instruction" or 
"CAnl." 

XI. A SMALL STATISTICAL EVALUATION 

Despite our de-emphasis of statistical evaluation 
for our purposes, we are going to present a small 
study of this nature in a specific instance with no 
attempt at broad generalization. Two sections of a 
course in General Psychology were taught by the 
same instructor, studied the same textbook and 
substantially the same subject matter, and at the end 
of the semester took the same identical final exam- 
ination. This examination was of the multiple-choice 
type. One section was taught in the fall of 1970 in a 
conventional classroom, while the other was taught 
in the fall of 1971 using regularly the Response System 
throughout the semester. The idea of comparing 
these two sections was more or less an after -thought. 
So there was no preplanning to make the two sections 
equal in all essential re.spects. Registrations for the 
sections were uncontrolled. Students, as well as the 
instructor, were not aware of the comparison. The 
instructor was enthusiastic about the use of the SRS, 
and most of the students who had such use assigned 
a high rating to its usefulness. 

We used SAT-Verbal and SAT-Math scores, 
English Composition score, "Converted Rank" in 
high school, and Grade Point Average at Skid more 
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*Beth Sulzer, Teaching- Educational Psychology with the Aid of a Computerized Student Response System. 1968, P. 6. 

tConceivably, students could sit on the floor, each carrying an Input Station box with a long wire connecting it 
with the scanning machine in the rear projection room, and the instructor's panel could be mounted on the arm 
of a sofa chair or coffee table. Let your imagination go! But whatever arrangements you choose, make sure 
that it will be easy to insure privacy of response. If there are rising tiers of seats, students on a higher 
level should not be able to see how a student on a lower tier responds* Similarly, the instructor's panel 
should be so placed that students cannot easily see which button the instructor pushes when* he puts in a cor- 
rect response to a question. All of this is not, strictly speaking, pertinent to an evaluation of the Response 
System, but colleges contemplating its installation may want to take notice of our errors on the side of 
formality and conventionality* 
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as measures of scholastic aptitude and academic 
performance to show the extent to which the two 
sections were equal. Then we compared the Final 
Examination Scores in General Psychology (not 
semester grades in that course) of the two sections 
in order to discover if there was any significant dif- 
ference between them. 

The following table summarizes the data: 



growth and general maturitj* might be an important 
factor. 

The differences between the-,means of the two 
sections in Psychology Pinal Examination Scores 
was 4. 2 in favor of tlie SRS Section. In all our 
statistical calculations, we assumed that the scores ' 
in the two sections were normally and independently 
distributed. The Standard Deviation of the difference 



TABLE J 

Showing the Characteristics of the Students in the Conventional Section and the SRS Section 
and Their Final Examination Scores in General Psychology 



Measures 



No. of Students* 
Freshman 
Sophomore 
Junior 

TOTAL 

SAT- Verbal Scores 
Mean 
S. D. 

SAT-Math Scores 
Mean 
S. D. 

Engl. Comp. Scores 
Mean 
S. D. 

Converted Rank in H. S. 
Mean 
S. D. 

Grade -Point -Ave rages 
at Skid more*** 
Mean 
S.D. 



Conventional 
Section 

22 (54%) 
12 (29%) 
7 (17%) 
41 

573 
63 

53B 
68 



586 
70 



60 

8 



2. 76 
.58 



SRS 
Section 

7 (20%) 
19 (54%) 

9 (26%) 
35 

558 
71 

542 
74 

584 
73 

59 
7 



3.03 
.50 



Difference: 
SRS over or 
under Conv'l 

-15 (-34%) 
+7 (+25%) 
+2 ( +9%) 
-6 

-15 
+8 

+4 
+6 

-2 

+3 

-1 
-1 



+.27 
-.08 



Psy. Final Exam Scores 
Mean 
S.D. 



76.4 

9 



80.4 
10 



+4.2 
+1 



*10 students from the conventional section and 5 students from the SRS 
section were excluded from the comparison fct^ lack of complete data. 
Those in the comparison were all females.' 
**Con version of Rank in High School into a two-digit ^ij^ure, taking class 
size into consideration, followed a formula developed by Validity Study 
Service of the College Entrance Examination Board. 
❖❖❖Grade-Point Average was not cumulative but only for the semester in 
which General Psychology was taken. 



Il vril) be noted that the two sections were not too 
dissimilar except in two respects. One was a less 
than 3% difference in SAT- Verbal scores in favor of 
the Conventional Section. The other was the fact that 
whereas 54% of the Conventional Section consisted of 
freshman students, in the SRS Section sophomores 
made up 54% of the class. An additional year of 



between means was found be 2. 2, and the 95% Con- 
fidence Interval for this difference was 4. 2 ± 4. 3, 
or from -0. 1 to 8. 5. By hypothesis testing, we 
found that the probability of a difference this great 
or greater happening by chance alone was 0.0574. 
So we are 94% sure that there is a significant dif- 
ference in the performance of the two sections. 



Could the difference be explained by the prepon- 
derance of sophomores in the SRS Section? To test 
this hypothesis, only sophomores in the two sections 
were compared. The data are summarized in the 
following table: 



Statistical studies seldom yield unequivocal con- 
clusions. It is true of this small study of ours also. 
Our findings are not out of line with conclusions of 
three similar studies previously made by other in- 
vestigators: 



TABLE 2 

Showing the Characteristics of only Sophomore Students in the Conventional Section and the SRS Section 
and Their Final Examination Scores in General Psychology 

Difference: 

Conventional SRS SRS over or 
_ Section Section under Conv4 



Measures 

No. of Sophomores 

SAT- Verbal Scores 
Mean 
S. D, 

SAT-Math Scores 
Mean 
S. D. 

Engl. Comp. Scores 
Mean 
S. D. 

Converted Rank in H. S. 
Mean 
S. D. 

Grade-Point -Averages 
at Skidmore 
Mean 
S.D. 



12 

584 
59 

529 
63 

566 
57 

60 
9 



19 

578 
64 

552 
65 

603 
57 

61 
7 



+7 

-6 
+5 

+23 
+2 

+37 
0 

+1 
-2 



2.66 
.51 



3.15 
.53 



+. 49 
+.02 



Psy. Final Exam. 
Mean 
S. D. 



Scores 



77.8 
10 



82.0 
10 



+4.2 
0 



This time there existed a less than 4% difference 
in SAT-Math Scores, a difference of about 6% in Eng- 
lish Composition Scores, and an 18% difference in 
Grade -Point-Average, all in favor of the SRS Secticr* 
The difference between the means of the two sections 
in Psychology Final Examination Scores was 4. 2, 
which was exactly the same as the difference between 
the two total sections. The Standard Deviation of the 
difference between the means of the two sophomore 
groups, by the small sample method, was 3.8, and 
the 95% Confidence Interval was 4. 2 ± 7. 77, or from 
-3. 57 to 11, 97. By hypothesis testing we are less 
than 75% sure that there is a significant difference 
in the performance of these two groups of sophomores. 
The slight superiority of the sophomores in the SRS 
Section in academic ability as indicated by their some- 
what better mean scores in SAT -Math, English Com- 
position, and Grade-Point -Average could at least 
partially explain some of the found difference between 
the means of the two groups. But at least the direc- 
tion of the difference is in favor of the SRS group. 

While a difference of 4. 2 points between the 
mean scores of the two sections on the Psychology 
final examination in favor of the SRS Section was not 



large, we must bear in mind an inherent limitation 
of the "equivalent group comparison" technique, 
namely, a common examination of the multiple- 
choice type had to be based on largely factual con- 
tent material taught in common to both sections and 
ignored any innovative materials and methods used 
only in the experimental section as well as the more 
subtle and subjective learnings that could have ac- 
companied the innovative features of the teaching- 
learning process. If there had been, instead, many 
small tests on specific topics in the course, designed 
in such a way as to reveal not only mastery of factual 
content but also more sophisticated learning outcomes, 
the difference in the performance of the two sections 
might have been larger or at least more meaningful, 
even if not any larger. 

(1> Dr. Henry O. Thompson at Syracuse University 
taught one section of a course in Religion in the tra- 
ditional way and another section using a much larger 
amount of visual material to replace some of his 
lectures and supplement others as well as employing 
the Student Response System. At the end of the 
semester it was found: 
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"Achievement, as measured by examin- 
ations, did not vary significantly between 
the groups. This finding is not definitive, 
however, since the final exam was based 
on the common lecture material and did 
not attempt to test the outcomes of the 
enriched experiences of the experimental 
group. "* 

(2) Dr. Martha W. Bradley, in teaching a course in 
Education, used a manual response method to a 
visual-verbal mode of instruction in one section and 
the Student Response System in another section. A 
comparison of student achievement in the two sections 
led to the conclusion: 

"The difference ( in favor of the SRS Section ) 
was significant (at the .01 level of confidence 
for the quizzes but not significant for the 
final examination)., "t 

(3) Dr. William P. Hillgartner compared the effects 
on student achievement of three treatments in three 
sections of a course in Teacher Education: 

(a) Use of SRS for individual diagnosis plus 
individualized assignments. 

(b) Use of SRS but no individualized assignments. 

(c) No use of SRS and no individualized assign- 
ments. 

He found: 

"The results strongly suggest that the 
regular incorporation of a response 
system into a program of instruction 
tends to raise the level of achievement. . . 
When the response system is used to collect 
feedback upon which individual assignments 
may be based, the level of achievement tends 
to rise even higher. "♦ 

XII. SUMMARY OP KVALUATION 

A large majority of our faculty users have found 
the Response System either "moderately" or "Quite" 
useful, in some cases, "very" useful, for the 
following purposes: 

1. Generating class discussion — breaking the ice, 
stimulating thought, preventmg defeneration of 
discussion into a lecture. 

2. Collecting information on student opinions, feel- 
, ings, attitudes, and other subjective aspects 



*Center for Instructional Communications. Mono- 
graph Series, II: Instructional Systems Develop- 
ment . Syracuse University. 1968. Part III, 
P. 4. 

tibid. Part III, P. 5. (italics inserted by the present 
writer. ) 

*lbid. Part III, P, 6. 



pervading the learning process whether in non- 
science or science courses. 

3. Student evaluation of instructional materials- - 
films, slides, readings, research, courses, 
instructors, etc. 

4. Monitoring student acquisition of basic know- 
ledge- -facts, concepts, procedures, etc. 
whether gained from individual study or from 
class instruction. 

5. Administering quizzes, tests, examinations 
whether for grading purposes or not. 

6. Monitoring individualized study and analysis of 
quantitative data entered through the SRS in the 
Free Mode. 

The chief limitation of the System is that it cannot 
accept constructed answers to open-ended questions. 
Hence, it cannot monitor the more sophisticated 
learning outcomes. However, even the most sophis- 
ticated learning has to rest on a foundation of basic 
knowledge of facts and concepts, which can be readily 
tapped by objective -type questions. If the instructor 
requires his students to assume greater responsibility 
for acquiring basic knowledge through individual 
study, this will free him from going over basic 
material in lectures and enable him to teach in class 
only what the students have failed to get themselves 
and to devote more time to sophisticated and creative 
teaching apart from the SRS. 

Obviously, it is unrealistic to expect any device 
to work perfectly in all situations. There is no 
panacea in the complicated task of college teaching. 
In the personal opinion of the Principal Investigator 
of this project, the outstanding virtue of the SRS is 
that it directs the attention of the instructor to student 
responses. Having associated himself with college 
teachers for over forty years in throe different 
countries, he believes that many instructors, in- 
cluding himself, when they walk into the classroom, 
are preoccupied with the material they are going to 
lecture on. Seldom do they think of student responses 
to the material unless they intend to tell a joke. Yet 
psychological researchers have demonstrated that 
students learn, not directly from instructional ma- 
terials, but from their own reactions to the teaching. 
This explains why different students often learn dif- 
ferent things from the same material or the same 
teacher. The instant collection of responses of all 
students and the speedy analysis .of the data by com- 
puter make it impossible for the instructor to ignore 
this evidence. He doesn»t wait days, weeks, or even 
months until he gives a test before he knows what the 
students have learned. With the evidence before him, 
he can continue to exploit his strengths, and where 
improvement is called for, he would probably try 
other ways of teaching. Thus, he is encouraged 
continuously to improve his instruction to the point of 
his own satisfaction. To use a Chinese simile, "The 
scratch is applied to exactly where it itches. " 
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